Federated search

ABSTRACT

Federated search is disclosed. A result associated with a search of a content source not natively configured to provide content type data for content items is received. A content type is associated with a content item included in the result based at least in part on information associated with the result.

CROSS REFERENCE TO OTHER APPLICATIONS

This application is a continuation of co-pending U.S. patent applicationSer. No. 13/103,644, entitled FEDERATED SEARCH filed May 9, 2011, whichis incorporated herein by reference for all purposes, which is acontinuation of U.S. patent application Ser. No. 11/393,217, now U.S.Pat. No. 7,962,464, entitled FEDERATED SEARCH filed Mar. 30, 2006 whichis incorporated herein by reference for all purposes.

BACKGROUND OF THE INVENTION

Computer system users can access data from many sources including localdata repositories, network attached data repositories, databases, andother resources (e.g., Internet websites). Some systems, for examplecontent management systems, are able to provide search services for itsusers; these services enable users to identify documents containing aparticular word, a particular phrase, a combination of words, related toparticular data associated with content (e.g., metadata). However, thisservice is typically only available for the content and metadata that isinternal to the search capable system (e.g., content that is managed bythe content management system). In some cases, search is possible withinother data sources, however only locally within systems or applicationsassociated with the other sources. It would be beneficial to be able totake advantage of the search ability of a system for searching contentor data that is either internal to the system, or also content or datathat is not internal to the system.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 is a block diagram illustrating an embodiment of a system forfederated search.

FIG. 2 is a flow diagram illustrating an embodiment of a process for afederated search.

FIG. 3 is a flow diagram illustrating an embodiment of a process forcausing results to be retrieved from the internal repository.

FIG. 4 is a flow diagram illustrating a process for causing results tobe retrieved from external repository(s) and external resource(s)including the retrieval of metadata regarding the results.

FIG. 5 is a flow diagram illustrating an embodiment of a process formigrating content on demand.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess, an apparatus, a system, a composition of matter, a computerreadable medium such as a computer readable storage medium or a computernetwork wherein program instructions are sent over optical or electroniccommunication links. In this specification, these implementations, orany other form that the invention may take, may be referred to astechniques. A component such as a processor or a memory described asbeing configured to perform a task includes both a general componentthat is temporarily configured to perform the task at a given time or aspecific component that is manufactured to perform the task. In general,the order of the steps of disclosed processes may be altered within thescope of the invention.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

Federated search is disclosed. A search is requested of a repository ofa second system not natively organized by content item type from a firstsystem that is organized by content item type. A search result andinformation associated with the search result are received. A type awaresearch result is created by adding type information to the search resultbased at least in part on the information associated with the searchresult. In some embodiments, the search is a desk top search tool thatintegrates searching of local sources as well as content managementrepositories, databases, other resources, etc. In some embodiments, theunified search does not require the requestor to know therepository/storage system's query language (e.g., a search for everydocument with “foo” for a system without full text search would receivea translated command that might include title contains “foo” or textcontains “foo”). In some embodiments, type data is associated with acontent source or portion thereof and the type data is associated with acontent item associated with (e.g., retrieved from and/or identified inthe course of a search of) the content source or portion thereof. Insome embodiments, a content item not stored in the internal repositorybut found in response to query performed via the internal repositoryand/or an associated content management system can be migrated to theinternal repository with an automatic capture of type information duringthe migration. In some embodiments, authentication is required for agiven external repository or resource, and a user is prompted forauthentication information (e.g., username and password, security token,etc.) as required when a session is initiated. When the search resultsare accessed again, the authentication information as well as path oraccess information is recalled from the prior search so that the resultsare requested again and/or access is provided without the user having tore-authenticate the session or recall the sequence of the search inorder to locate the desired information (e.g., search results oftenprovide a temporary path to requested information and in this case thefederated search system provides in a manner transparent to the user theability to re-access the information without the concern that atemporary path may have expired).

FIG. 1 is a block diagram illustrating an embodiment of a system forfederated search. In the example shown, one or more users connect toweb-based content management application 102 using a computer system,represented in FIG. 1 by computer 100. In the example shown, web-basedcontent management application 102 is built on a content managementsystem vendor-provided web development kit (WDK) 104. In someembodiments, WDK 104 comprises a set of tools to facilitate developmentof web-based applications, e.g., by providing reusable componentspre-configured to interact with a content server and/or other repositoryto provide content management services. Web-based content managementapplication 102 in some embodiments provides to users content type awarecontent management services and/or functions. An example of a contenttype aware service or function is providing a view (display) based atleast in part on content item type, e.g., displaying a “calendar” as agrid and an email message with “to”, “from”, “subject”, and/or otherfields in a header display region at the top of the display, attachmentnames/icons in an attachment display region below the header, andmessage text in a message body display region at the bottom; enabling auser to search by content type; and displaying query results in a mannerdetermined based at least in part on content type. Web-based contentmanagement application 102 also includes search services 106. In someembodiments, a user requests a search to a content management system.The content management service searches both the internal repositoriesof the content management system and external repositories and otherattached resources. The content management system receives the searchrequest and the WDK 104 translates the search request for searchservices 106. Search services 106 translates the search request for adirect search of internal content management system repository 108 andan indirect search of external repositories/resources via enterprisecontent integration server 110. In some embodiments, search services 106are provided by business logic and/or foundation classes that comprise acontent management framework on which web-based content managementapplication 102 and WDK 104 are based. Enterprise content integration(ECI) server 110 targets the search request for a search of one or moreexternal repositories, represented in FIG. 1 by 114 and 118, and/orother resources, represented in FIG. 1 by 122. In various embodiments,external repositories comprise other content management systems,database systems, legacy content management systems not capable of thetype of search desired, (e.g., Filenet™, LexisNexis™, Lotus Notes™,etc.). In various embodiments, other resources include informationsources or search services—for example, Google™, Yahoo™, MSN™, MySimon™,Pricewatch™, AskJeeves™, etc. The targeted search request is translatedand adapted using a configuration adapter, represented in FIG. 1 byconfiguration adapter 112, 116, and 120. The configuration adapter alsotranslates and adapts the responses to the search request and passes theresponses to ECI server 110. In some embodiments, the configurationadapter includes associated information in the response to the searchrequest. In some embodiments, the associated information is ultimatelyused as metadata information that enables the content management systemto treat the search results as typed information for the results of thesearch. In some embodiments, a query broker (not shown) at the contentmanagement system is configured to communicate with the ECI server 110and to use information about the external content sources accessed viaECI server 110 to associate type information with results received fromECI server 110. The results from ECI server 110 in some embodimentsindicate the external content source for each responsive content item(or group of items), and the query broker at the content managementsystem associates type information with each content item (or set ofcontent items) based on the source, as identified for example by dataassociated with a configuration file and/or adapter used to retrieveand/or identify the content from the external source. In someembodiments, more than one configuration adapter is associated with aspecific repository or resource—for example, one configuration adapteris configured to facilitate searching for contract information in aspecific database and another configuration adapter is configured tofacilitate searching for financial information or specificationinformation in the same specific database. In some embodiments, aconfiguration adapter is easily configured by a user, systemadministrator, system manager using an application with a graphical userinterface (e.g., a WYSIWYG editor) to establish general parsing andsyntax rules for communication between the ECI server and the externalrepository or resource. In various embodiments, the configurationadapter configuration is based on preconfigured templates, has a selflearning engine in order to self-configure based at least in part on theexternal repository type and/or external resource type.

FIG. 2 is a flow diagram illustrating an embodiment of a process for afederated search. In various embodiments, the process of FIG. 2 isexecuted at least in part in web-based content management application102, WDK 104, and/or search services 106 of FIG. 1. In the exampleshown, in 200 a query is received regarding content. In someembodiments, the query requests information regarding the content usingtype information—for example, a request is made to locate all contractsassociated with a company name. In 202, the query is split for theinternal repository and for external repository(s) and/or externalresource(s). In some embodiments, the query is translated from a userrequest to a request that can be understood by search services (e.g.,WDK 104 translates the query for search services 106—for example, auniversal search syntax/language such as DFC). In some embodiments,search services 106 translates the query so that the query isappropriate for the internal repository and the federated search broker,which interacts through configuration adapters, with externalrepository(s) and external resource(s). In some embodiments, thetranslation comprises the generation of a more general and degradedquery that can be processed by the external repository. The translationcan compensate for a lack of sophistication of the target systems andcan be combined with a post-filtering step to ensure the correctness ofthe results. In 204, the results are caused to be retrieved from theinternal repository. In some embodiments, the translated query for theinternal repository (e.g., a proprietary query language for the internalrepository or an API call) is caused to execute and the results arereceived from the internal repository which natively understands typedinformation. In 206, the results are caused to be retrieved fromexternal repository(s) and other external resource(s), as applicable,including the retrieval of metadata regarding the results. In someembodiments, the translated query for the external repository(s) and/orexternal resource(s) (e.g., a syntax that conforms to an API or otherinterface or language associated with enterprise content integration(ECI) server 110) is caused to execute and the results are received fromexternal repository(s) and/or resource(s) which do not nativelyunderstand type information. In 208, type information is associated withthe results based at least in part on the metadata. In some embodiments,type information is associated with the results based on metadatainformation received along with results. In some embodiments, aconfiguration adapter that is configured, on the one hand, to adaptsearch queries for an external repository or external resource from anECI server, and, on the other hand, to adapt search query results froman external repository or external resource to an ECI server includingadding, querying for, and/or generating appropriate associated metadatafor the query results. In some embodiments, the type information doesnot accompany the search results and instead is learned prior to and/orsubsequent to the search being performed and/or results returned. Insome embodiments, 206 and 208 are performed by a federated query brokerat the content management system, configured to translate queries into alanguage and/or format understood by the ECI server 110 and to translateresults from ECI server 110 into a form and format understood and usableat the content management system. In some embodiments, the functions ofthe federated query broker as described in the preceding sentence areperformed by a broker on ECI server 110 and/or integrated into thefunctionality of ECI server 110. For example, in some embodiments theECI server 110 is configured to use information associated with anexternal content source, e.g., a configuration adapter and/or associatedconfiguration file associated with the external content source, toassociate type information with content items responsive to a query,e.g., by injecting content type data (e.g., metadata) as appropriate ina stream of results data sent to the content management system fromwhich a federated query was received. In some embodiments, resultsreceived from external sources are post-filtered to check and/or ensurethat the results are relevant to and/or valid for the original query asexpressed in 200. Post-filtering can be important in the event that theoriginal query is translated in such a manner as to make the query moregeneral or if the specificity of the query has been degraded. In 210,the results from the internal repository and external repository(s) andexternal resource(s) are combined and presented as typed information. Insome embodiments, transparent to a requesting user, combined searchresults from the internal repository and external repository(s) and/orexternal resource(s) are presented to a user in a content type awaremanner as a single/integrated set of results. In some embodiments, theresults are deduplicated (i.e., results indicating identical contentitems are reduced to one result) using one or more of the following: aunique identifier or a characteristic attribute such as size, date ofcreation, type, last modification date, etc. In various embodiments, thesearch results from the internal repository, the search results from theexternal repository(s), the results from the external resource(s), orany combination are deduplicated. In some embodiments, content data orresult data are translated from a native format associated with theexternal content source into a common format associated with the contentmanagement system (e.g., XML, text, etc.). In some embodiments, contentdata or result data are not translated from a native format associatedwith the external content source (e.g., Microsoft Word™ format, AdobePDF™ format, etc.).

FIG. 3 is a flow diagram illustrating an embodiment of a process forcausing results to be retrieved from the internal repository. In someembodiments, the process of FIG. 3 is used to implement 206 of FIG. 2.In the example shown, in 300 the query is caused to be executed. In someembodiments, the search service of a content management system whichnatively understands typed information is used to perform a search ofthe internal repository by causing a query to be executed by therepository. In 302, typed results are received. The results from theinternal repository can include information regarding content type—forexample, information arising from associated metadata information withrespect to a specific content item.

FIG. 4 is a flow diagram illustrating a process for causing results tobe retrieved from external repository(s) and external resource(s)including the retrieval of metadata regarding the results. In someembodiments, the process of FIG. 4 is used to implement 208 of FIG. 2.In the example shown, in 400 the query is caused to be split for eachexternal repository and/or external resource. In some embodiments, thesearch is performed only with respect to a subset of external contentsources accessible via the ECI server, such as only those externalcontent sources that include content of a content type identified in thequery. In some embodiments, the ECI server determines which externalsources to search, e.g., based on one or more search terms (e.g.,content type). In some embodiments, a broker at the content managementsystem determines which external sources to search and identifies to theECI server the external content sources to be searched for a particularquery. In some embodiments, authentication is required for a givenexternal repository or resource, and a user is prompted forauthentication information as required when a session is initiated. Whenthe search results are accessed again, the authentication information aswell as path or access information is recalled from the prior search sothat the results are presented again or access is provided without theuser having to re-authenticate the session or recall the sequence of thesearch in order to locate the desired information. In 402, for eachexternal repository and/or resource, a translation of the query iscaused for the specific external repository or resource. In someembodiments, a configuration adapter is configured to translate thequery for a specific repository or resource. In 404, for each splitquery, the query is caused to be executed. In 406, for each split query,the results are caused to be translated. In 408, all results from eachof the split queries are received including metadata. In someembodiments, the metadata is a collection of attribute data from theexternal repository or resource associated with the search result. Insome embodiments, the search result is translated appropriately for theECI server and the search service.

FIG. 5 is a flow diagram illustrating an embodiment of a process formigrating content on demand. In some embodiments, the process of FIG. 5follows the process of FIG. 2. Federated search results if not a part ofthe internal repository can be migrated into the internal repository andstored as typed content with associated metadata. In the example shown,in 500 content is indicated as being not in the internal repository. Insome embodiments, the results of the federated search are presented tothe search requestor in a manner that indicates that the search resultis from an external repository or resource—for example, displayed in adifferent color (e.g., different from the color used to display a searchresult from the internal repository), with a different background (e.g.,different from the background used to display a search result from theinternal repository), with an additional or different icon (e.g.,different from the icon used to display a search result from theinternal repository), in a separate result window, in a separate list,or by any other appropriate manner of indicating. In 502, an indicationis received of the content items that are to be migrated into theinternal repository. In various embodiments, the indication comprises aselection of items in a list by highlighting in a list, clicking radiobuttons, checking boxes, typing in a text window, or any otherappropriate manner of indicating. In various embodiments, the one ormore content items are retrieved along with associated metadata in orderto be able to load the content item and associated metadata into theinternal database so that content is appropriately type or the contentitems and associated metadata are already loaded in such a manner as toenable loading the content item and associated metadata into theinternal database. In 504, object(s) are created in the repository,e.g., one or more persistent objects each configured to represent acorresponding content item in the repository, e.g., by storing metadataabout the content item and/or making available one or more methods foraccessing and/or managing the content item. In some embodiments, anobject is created for each content item to be migrated. In someembodiments, the content items are checked to ensure that there are noduplicates being stored in the repository including checking forduplicates in the new information and/or checking to see if the newinformation duplicates any previously stored information. In variousembodiments, checking for duplicates comprises checking a file size,checking a file creation or modification date, checking a checksum,checking an attribute, or any other appropriate manner of checking tosee if a content item is a duplicate. In 506, metadata is loaded (e.g.,by populating attributes of the object(s) created at 504) and content isstored in the repository. In some embodiments, the metadata includescontent type information derived from metadata associated with thecontent item, content data comprising the content item, and/or typeinformation associated with and/or received from a configuration adapterand/or file associated with the external content source from which thecontent item was imported. In some embodiments, metadata and/or aportion thereof is displayed to a user for verification and/or editingprior to saving the object to the repository, e.g., to permit a user tocorrect an error in mapping content item metadata received from and/orassociated with the external source—e.g., content type—to attributes ofthe object being configured to represent the content item in the localrepository. In some embodiments, the search result, sessionauthentication information, temporary path information (e.g., a URL froman external resource search—for example, a Google™ search result URL) isstored as metadata associated with a content item.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

1. A system for managing content comprising: A processor configured to:receive at a content management system that is organized by content itemtype a search query using a content type data to request one or morecontent items; cause the search query to search in an external contentsource that is not natively configured to provide content type data forcontent items; receive at the content management system a resultassociated with the search query from a search of the external contentsource not natively configured to provide content type data for contentitems; associate a content type with a content item included in theresult based at least in part on an information associated with theresult; and provide the result associated with the search query of theexternal content source; and a memory coupled to the processor andconfigured to provide the processor with instructions.
 2. A system as inclaim 1, wherein the processor is further configured to receive theinformation associated with the result associated with the search queryof the external content source.
 3. A system as in claim 1, wherein theinformation associated with the result associated with the search queryof the external content source includes metadata associated with thecontent item.
 4. A system as in claim 1, the information associated withthe result associated with the search query of the external contentsource includes information about the content source.
 5. A system as inclaim 1, the information associated with the result associated with thesearch query of the external content source includes informationassociating the content type with the content source.
 6. A system as inclaim 1, wherein the external content source comprises an externalsource of content that is external to the content management system. 7.A system as in claim 1, wherein the external content source comprisesone of the following: a content management system, a database system, alegacy content management system, an information source system, or asearch services system.
 8. A system as in claim 1, wherein causing thesearch query to search in an external content source includestranslating the request into a form associated with the external contentsource.
 9. A system as in claim 8, wherein the request comprises a moregeneral request.
 10. A system as in claim 9, wherein receiving theresult associated with the search query of the external content sourceincludes post-filtering the result to check that the results arerelevant to the requested search of the content source.
 11. A system asin claim 1, wherein causing the search query to search in the externalcontent source includes providing authentication information to theexternal content source.
 12. A system as in claim 1, wherein receivingthe result associated with the search query of the external contentsource includes translating the result into a form associated with thecontent management system with respect to which the external contentsource comprises an external source.
 13. A system as in claim 1, whereinthe result associated with the search query of the external contentsource includes information enabling the search result associated withthe search query of the external content source to be obtained againfrom the external content source.
 14. A system as in claim 1, whereinreceiving the result associated with the search query of the externalcontent source includes one or more of the following: adding, queryingfor, or generating the information associated with the result.
 15. Acomputer program product for managing content, the computer programproduct being embodied in a computer readable storage medium andcomprising computer instructions for: receiving at a content managementsystem that is organized by content item type a search query using acontent type data to request one or more content items; causing thesearch query to search in an external content source that is notnatively configured to provide content type data for content items;receiving at the content management system a result associated with thesearch query from a search of the external content source not nativelyconfigured to provide content type data for content items; associating acontent type with a content item included in the result based at leastin part on an information associated with the result; and providing theresult associated with the search query of the external content source.16. A method for managing content comprising: receiving at a contentmanagement system that is organized by content item type a search queryusing a content type data to request one or more content items; causingthe search query to search in an external content source that is notnatively configured to provide content type data for content items;receiving at the content management system a result associated with thesearch query from a search of the external content source not nativelyconfigured to provide content type data for content items; associating acontent type with a content item included in the result based at leastin part on an information associated with the result; and providing theresult associated with the search query of the external content source.